docs/examples/Validation Examples.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "89cf2628",
   "metadata": {},
   "source": [
    "# Validation Examples\n",
    "\n",
    "As a result of optimizations, and the low level nature of the Power Grid Model's mathematical core, the core exceptions may not always be clear to the user. Therefore an optional validation mechanism is supplied, which validates data structures and values off-line. It is recommended to always validate your data before constructing a PowerGridModel instance. An alternative approach would be to validate only when an exception is raised, but be aware that not all data errors will raise exceptions, most of them wil just yield invalid results without warning."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3cbdd654",
   "metadata": {},
   "source": [
    "The basic methods and class definitions are available in the `power_grid_model.validation` module:\n",
    "\n",
    "```python\n",
    "# Manual validation\n",
    "#   validate_input_data() assumes that you won't be using update data in your calculation.\n",
    "#   validate_batch_data() validates input_data in combination with batch/update data.\n",
    "validate_input_data(input_data, calculation_type, symmetric) -> List[ValidationError]\n",
    "validate_batch_data(input_data, update_data, calculation_type, symmetric) -> Dict[int, List[ValidationError]]\n",
    "\n",
    "# Assertions\n",
    "#   assert_valid_input_data() and assert_valid_batch_data() raise a ValidationException,\n",
    "#   containing the list/dict of errors, when the data is invalid.\n",
    "assert_valid_input_data(input_data, calculation_type, symmetric) raises ValidationException\n",
    "assert_valid_batch_data(input_data, calculation_type, update_data, symmetric) raises ValidationException\n",
    "\n",
    "# Utilities\n",
    "#   errors_to_string() converts a set of errors to a human readable (multi-line) string representation\n",
    "errors_to_string(errors, name, details)\n",
    "```\n",
    "\n",
    "Each validation error is an object which can be converted to a compact human-readable message using `str(error)`. It\n",
    "contains three member variables `component`, `field` and `ids`, which can be used to gather more specific information about the validation error, e.g. which object IDs are involved.\n",
    "\n",
    "```python\n",
    "class ValidationError:\n",
    "    \n",
    "    # Component(s): e.g. \"node\" or [\"node\", \"line\"]\n",
    "    component: Union[str, List[str]]\n",
    "    \n",
    "    # Field(s): e.g. \"id\" or [\"line_from\", \"line_to\"] or [(\"node\", \"id\"), (\"line\", \"id\")]\n",
    "    field: Union[str, List[str], List[Tuple[str, str]]]\n",
    "\n",
    "    # IDs: e.g. [1, 2, 3] or [(\"node\", 1), (\"line\", 1)]\n",
    "    ids: Union[List[int], List[Tuple[str, int]]] = []    \n",
    "    \n",
    "```\n",
    "\n",
    "Note: The data types of `input_data` and `update_data` are the same as expected by the power grid model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "d122ee22",
   "metadata": {},
   "outputs": [],
   "source": [
    "from power_grid_model import PowerGridModel, initialize_array\n",
    "\n",
    "# A power grid containing several errors\n",
    "\n",
    "# node\n",
    "node_error = initialize_array(\"input\", \"node\", 3)\n",
    "node_error[\"id\"] = [1, 2, 3]\n",
    "node_error[\"u_rated\"] = [10.5e3]\n",
    "\n",
    "# line\n",
    "line_error = initialize_array(\"input\", \"line\", 3)\n",
    "line_error[\"id\"] = [4, 5, 6]\n",
    "line_error[\"from_node\"] = [1, 2, 3]\n",
    "line_error[\"to_node\"] = [2, 3, 4]\n",
    "line_error[\"from_status\"] = [True]\n",
    "line_error[\"to_status\"] = [True]\n",
    "line_error[\"r1\"] = [0.25]\n",
    "line_error[\"x1\"] = [0.2]\n",
    "line_error[\"c1\"] = [10e-6]\n",
    "line_error[\"tan1\"] = [0.0]\n",
    "\n",
    "# Power Sensor\n",
    "sensor_error = initialize_array(\"input\", \"sym_power_sensor\", 2)\n",
    "sensor_error[\"id\"] = [6, 7]\n",
    "sensor_error[\"measured_object\"] = [3, 4]\n",
    "sensor_error[\"measured_terminal_type\"] = [0, 2]\n",
    "sensor_error[\"p_measured\"] = [0]\n",
    "sensor_error[\"q_measured\"] = [0]\n",
    "sensor_error[\"power_sigma\"] = [0]\n",
    "\n",
    "error_data = {\"node\": node_error, \"line\": line_error, \"sym_power_sensor\": sensor_error}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "d997e738",
   "metadata": {},
   "outputs": [
    {
     "ename": "PowerGridError",
     "evalue": "Wrong type for object with id 4\n\nTry validate_input_data() or validate_batch_data() to validate your data.\n",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mPowerGridError\u001b[0m                            Traceback (most recent call last)",
      "Cell \u001b[1;32mIn[2], line 2\u001b[0m\n\u001b[0;32m      1\u001b[0m \u001b[38;5;66;03m# Without validation\u001b[39;00m\n\u001b[1;32m----> 2\u001b[0m model \u001b[38;5;241m=\u001b[39m \u001b[43mPowerGridModel\u001b[49m\u001b[43m(\u001b[49m\u001b[43merror_data\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m      3\u001b[0m output_data \u001b[38;5;241m=\u001b[39m model\u001b[38;5;241m.\u001b[39mcalculate_state_estimation(symmetric\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mTrue\u001b[39;00m)\n",
      "File \u001b[1;32mC:\\src\\power-grid-model\\venv\\Lib\\site-packages\\power_grid_model\\core\\power_grid_model.py:108\u001b[0m, in \u001b[0;36mPowerGridModel.__init__\u001b[1;34m(self, input_data, system_frequency)\u001b[0m\n\u001b[0;32m    106\u001b[0m prepared_input \u001b[38;5;241m=\u001b[39m prepare_input_view(input_data)\n\u001b[0;32m    107\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_model_ptr \u001b[38;5;241m=\u001b[39m pgc\u001b[38;5;241m.\u001b[39mcreate_model(system_frequency, input_data\u001b[38;5;241m=\u001b[39mprepared_input\u001b[38;5;241m.\u001b[39mget_dataset_ptr())\n\u001b[1;32m--> 108\u001b[0m \u001b[43massert_no_error\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    109\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_all_component_count \u001b[38;5;241m=\u001b[39m {k: v \u001b[38;5;28;01mfor\u001b[39;00m k, v \u001b[38;5;129;01min\u001b[39;00m prepared_input\u001b[38;5;241m.\u001b[39mget_info()\u001b[38;5;241m.\u001b[39mtotal_elements()\u001b[38;5;241m.\u001b[39mitems() \u001b[38;5;28;01mif\u001b[39;00m v \u001b[38;5;241m>\u001b[39m \u001b[38;5;241m0\u001b[39m}\n",
      "File \u001b[1;32mC:\\src\\power-grid-model\\venv\\Lib\\site-packages\\power_grid_model\\core\\error_handling.py:70\u001b[0m, in \u001b[0;36massert_no_error\u001b[1;34m(batch_size)\u001b[0m\n\u001b[0;32m     68\u001b[0m error \u001b[38;5;241m=\u001b[39m find_error(batch_size\u001b[38;5;241m=\u001b[39mbatch_size)\n\u001b[0;32m     69\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m error \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[1;32m---> 70\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m error\n",
      "\u001b[1;31mPowerGridError\u001b[0m: Wrong type for object with id 4\n\nTry validate_input_data() or validate_batch_data() to validate your data.\n"
     ]
    }
   ],
   "source": [
    "# Without validation\n",
    "model = PowerGridModel(error_data)\n",
    "output_data = model.calculate_state_estimation(symmetric=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "fd84be3c",
   "metadata": {},
   "outputs": [
    {
     "ename": "ValidationException",
     "evalue": "There are 5 validation errors in input_data:\n   1. Fields line.id and sym_power_sensor.id are not unique for 2 lines/sym_power_sensors.\n   2. Field 'to_node' does not contain a valid node id for 1 line.\n   3. Field 'power_sigma' is not greater than zero for 2 sym_power_sensors.\n   4. Field 'measured_object' does not contain a valid line/transformer id for 1 sym_power_sensor. (measured_terminal_type=branch_from)\n   5. Field 'measured_object' does not contain a valid source id for 1 sym_power_sensor. (measured_terminal_type=source)",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mValidationException\u001b[0m                       Traceback (most recent call last)",
      "Cell \u001b[1;32mIn[3], line 4\u001b[0m\n\u001b[0;32m      1\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mpower_grid_model\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mvalidation\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m assert_valid_input_data\n\u001b[0;32m      3\u001b[0m \u001b[38;5;66;03m# Assert valid data\u001b[39;00m\n\u001b[1;32m----> 4\u001b[0m \u001b[43massert_valid_input_data\u001b[49m\u001b[43m(\u001b[49m\u001b[43merror_data\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43msymmetric\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43;01mTrue\u001b[39;49;00m\u001b[43m)\u001b[49m\n\u001b[0;32m      5\u001b[0m model \u001b[38;5;241m=\u001b[39m PowerGridModel(error_data)\n\u001b[0;32m      6\u001b[0m output_data \u001b[38;5;241m=\u001b[39m model\u001b[38;5;241m.\u001b[39mcalculate_state_estimation(symmetric\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mTrue\u001b[39;00m)\n",
      "File \u001b[1;32mC:\\src\\power-grid-model\\venv\\Lib\\site-packages\\power_grid_model\\validation\\assertions.py:57\u001b[0m, in \u001b[0;36massert_valid_input_data\u001b[1;34m(input_data, calculation_type, symmetric)\u001b[0m\n\u001b[0;32m     53\u001b[0m validation_errors \u001b[38;5;241m=\u001b[39m validate_input_data(\n\u001b[0;32m     54\u001b[0m     input_data\u001b[38;5;241m=\u001b[39minput_data, calculation_type\u001b[38;5;241m=\u001b[39mcalculation_type, symmetric\u001b[38;5;241m=\u001b[39msymmetric\n\u001b[0;32m     55\u001b[0m )\n\u001b[0;32m     56\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m validation_errors:\n\u001b[1;32m---> 57\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m ValidationException(validation_errors, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124minput_data\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n",
      "\u001b[1;31mValidationException\u001b[0m: There are 5 validation errors in input_data:\n   1. Fields line.id and sym_power_sensor.id are not unique for 2 lines/sym_power_sensors.\n   2. Field 'to_node' does not contain a valid node id for 1 line.\n   3. Field 'power_sigma' is not greater than zero for 2 sym_power_sensors.\n   4. Field 'measured_object' does not contain a valid line/transformer id for 1 sym_power_sensor. (measured_terminal_type=branch_from)\n   5. Field 'measured_object' does not contain a valid source id for 1 sym_power_sensor. (measured_terminal_type=source)"
     ]
    }
   ],
   "source": [
    "from power_grid_model.validation import assert_valid_input_data\n",
    "\n",
    "# Assert valid data\n",
    "assert_valid_input_data(error_data, symmetric=True)\n",
    "model = PowerGridModel(error_data)\n",
    "output_data = model.calculate_state_estimation(symmetric=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "6286ece0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MultiComponentNotUniqueError ['line', 'sym_power_sensor'] : [('line', 6), ('sym_power_sensor', 6)]\n",
      "InvalidIdError line : [6]\n",
      "NotGreaterThanError sym_power_sensor : [6, 7]\n",
      "InvalidIdError sym_power_sensor : [6]\n",
      "InvalidIdError sym_power_sensor : [7]\n"
     ]
    }
   ],
   "source": [
    "from power_grid_model.validation import ValidationException\n",
    "\n",
    "# Assert valid data and display component ids\n",
    "try:\n",
    "    assert_valid_input_data(error_data, symmetric=True)\n",
    "    model = PowerGridModel(error_data)\n",
    "    output_data = model.calculate_state_estimation(symmetric=True)\n",
    "except ValidationException as ex:\n",
    "    for error in ex.errors:\n",
    "        print(type(error).__name__, error.component, \":\", error.ids)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "1e4ca721",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There are 5 validation errors in the data:\n",
      "   1. Fields line.id and sym_power_sensor.id are not unique for 2 lines/sym_power_sensors.\n",
      "   2. Field 'to_node' does not contain a valid node id for 1 line.\n",
      "   3. Field 'power_sigma' is not greater than zero for 2 sym_power_sensors.\n",
      "   4. Field 'measured_object' does not contain a valid line/transformer id for 1 sym_power_sensor. (measured_terminal_type=branch_from)\n",
      "   5. Field 'measured_object' does not contain a valid source id for 1 sym_power_sensor. (measured_terminal_type=source)\n"
     ]
    }
   ],
   "source": [
    "from power_grid_model.validation import validate_input_data, errors_to_string\n",
    "\n",
    "# Validation only as exception handling\n",
    "try:\n",
    "    model = PowerGridModel(error_data)\n",
    "    output_data = model.calculate_state_estimation(symmetric=True)\n",
    "except RuntimeError as ex:\n",
    "    errors = validate_input_data(error_data, symmetric=True)\n",
    "    print(errors_to_string(errors))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "91a1aff6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There are 5 validation errors in the data:\n",
      "\n",
      "\tFields line.id and sym_power_sensor.id are not unique for 2 lines/sym_power_sensors.\n",
      "\t\tcomponent: line/sym_power_sensor\n",
      "\t\tfield: line.id and sym_power_sensor.id\n",
      "\t\tids: [('line', 6), ('sym_power_sensor', 6)]\n",
      "\n",
      "\tField 'to_node' does not contain a valid node id for 1 line.\n",
      "\t\tcomponent: line\n",
      "\t\tfield: 'to_node'\n",
      "\t\tids: [6]\n",
      "\t\tref_components: node\n",
      "\t\tfilters: \n",
      "\n",
      "\tField 'power_sigma' is not greater than zero for 2 sym_power_sensors.\n",
      "\t\tcomponent: sym_power_sensor\n",
      "\t\tfield: 'power_sigma'\n",
      "\t\tids: [6, 7]\n",
      "\t\tref_value: zero\n",
      "\n",
      "\tField 'measured_object' does not contain a valid line/transformer id for 1 sym_power_sensor. (measured_terminal_type=branch_from)\n",
      "\t\tcomponent: sym_power_sensor\n",
      "\t\tfield: 'measured_object'\n",
      "\t\tids: [6]\n",
      "\t\tref_components: line/transformer\n",
      "\t\tfilters: (measured_terminal_type=branch_from)\n",
      "\n",
      "\tField 'measured_object' does not contain a valid source id for 1 sym_power_sensor. (measured_terminal_type=source)\n",
      "\t\tcomponent: sym_power_sensor\n",
      "\t\tfield: 'measured_object'\n",
      "\t\tids: [7]\n",
      "\t\tref_components: source\n",
      "\t\tfilters: (measured_terminal_type=source)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Manual checking and display detailed information about the invalid data\n",
    "errors = validate_input_data(error_data, symmetric=True)\n",
    "print(errors_to_string(errors, details=True))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.2"
  },
  "vscode": {
   "interpreter": {
    "hash": "39f035c5e58f53e973c1999fdd4393d6db2dd3f04fe05f524988f90f4bbdb576"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}